Aeolian sand dunes are prevalent landforms on the surface of Mars. While attempts have been made to map their distribution in satellite images, it is likely that many remain unmapped. In this project, we seek to develop a method to automatically identify dune fields on the surface of mars using deep learning. Specifically, we hope to achieve this through an image segmentation approach use of a convolutional neural (UNet) via a collection of images captured by the Mars Reconnaissance Orbiter.
![]() | ![]() |
Left. Image of the Red Planet (Credit: NASA). Right. False-color image shows 'sea' of dunes near Mars' northern polar cap (Credit: NASA).
![]() | ![]() |
Left. Image captured by the Mars Reconnaisance Orbiter, shows barchan dunes (Credit: NASA/JPL/University of Arizona/USGS).Right. First dunes studied up-close, known as "High Dune" on NASA's Curiosity rover (Credit:NASA/JPL-Caltech).
Deposits on the surface of Mars provide the only available record of its temporal evolution. Many relict features remain well preserved on the Martian surface, documenting periods of actively flowing water and Martian wind regimes. One such landform-type are aeolian dunes, which form and evolve due to wind-driven sediment transport. A variety of dune types including those commonly seen on Earth (barchan, linear, cresentic) are also observed on the Martian surface. However, a variety of dune geometries exist on Mars that are not found on Earth. The distribution, configuration, and types of dunes on Mars provide history of changes on the Red Planet. NASA has even developed a way of monitoring dune movement throught time across the Martian surface!

We would like to develop an automated approach for detecting and classifying martian dune fields through the application of deep learning and image segmentation. We will do so through the use of a convolutional neural network trained using hand-mapped dune field locations and high-resolution images captured by the THEMIS IR camera on the Mars Reconnaisance Orbitor.
The Mars Reconnaisance Orbitor provides a rich dataset of visible imagery and thermal data from the Martian surface. The Thermal Emission Imaging System (THEMIS) IR camera affixed to the Mars Reconnaisance Orbiter provides the data used in this study. A global mosaic of THEMIS imagery sampled at 100x100m per pixel resolution is the primary dataset for model training, validation, and testing.
![]() | ![]() |
Left. Mars Reconnaisance Orbiter, THEMIS camera imagery acquisition (Credit: NASA). Right. Global THEMIS Mosaic has 100x100m per pixel spatial resolution (Credit: NASA).
Previous workers have mapped the locations of dune fields as polygons based on visual inspection of imagery from the Mars Reconnaisance Oribiter. These data and information on the location of mapped Martian dune fields have been compiled into the Mars Global Digital Dune Database: MC2–MC29. However, many dune fields on the Martian surface remain unmapped.

Deep learning, and specifically convolutional neural networks, in the scope of planetary surfaces and surface processes has been applied primarily to automatically identify and measure craters on both the Moon and Mars (Lee, 2019; Silbert et al., 2019). However, automatic detection and classification of dune fields using CNNs has not been achieved.
We decided to use the UNet architecture to approach our problem after seeing its success in identifying and mapping craters on the Moon and Mars (DeepMoon and DeepMars Github repositories)(Lee, 2019; Silbert et al., 2019), which proved successful to determine the positions and sizes of craters from Lunar and Martian digital elevation maps.
Their UNet convolutional neural network (and by design our neural network) is provided in the schematic below.


In our project we implement a custom version of the UNET architecture (Ronneberger et al., 2015). This architecture consists of a contracting path (left side) and expansive path (right side), joined through multi-level skip connections (middle). Martian THEMIS images are input to the contracting path and predictions are output from a final layer following the expansive path.
In the above schematic, boxes represent cross-sections of square feature maps. Each map's dimensions are indicated on its lower left, and its number of channels are indicated above it. In this diagram, the leftmost map is a 256 × 256 grayscale image sampled from the digital elevation map of the Moon, and the rightmost the CNN's binary ring mask prediction. We employ the same approach for highlighting regions that correspond to dune fields in the THEMIS imagery.
Prior to model construction, we visualize the data using mapping software (QGIS). We see that the dunes are strikingly different than the landscape surrounding them. We also notice that many dune fields are located within craters.
Dune fields are clearly imaged by the THEMIS IR camera. Because the images are high-resolution (100x100m per pixel) individual dunes are clearly distinguishable.
However, we observe a wide degree of variability in dune field geometry and prevalence. We will have to see if the CNN can identify this variability!
Training images were derived from the THEMIS Day IR 100m Global Mosaic, which provides imagery across the entire Martian surface at 100 meters per pixel. The full imagery dataset is ~22Gb
Hand-mapped dune fields constitute the label data for this study. These data were obtained from the Global Mars Dune Database, in which 550 dune fields have been mapped by hand based on visual interpretation of satellite imagery.
A significant amount of time for this project went into the data collection, cleaning and preparation.
We first reprojected the label shapefiles in QGIS to ensure consistent coordinate reference systems between the labels and images. Mars Simple Cylindrical coordinates were used.
A binary raster was generated from the label feature class where dune field polygon pixels have values of 1 while other pixels have Null values.
Due of the file size of the original THEMIS mosaic (~20Gb), we first cropped the satellite image and label rasters to a region which included the majority of the mapped dune field locations See Above.
We then generated image and label 'tiles' with dimensions of 256x256 pixels using a series of python scrips in combination with the open-source GNU IMAGE MANIPULATION PROGRAM (GIMP). These scripts can be found on Github.
This workflow resulted in 78,337 image and label tiles. For training purposes, we filtered the image tiles to include only images that contained at least one (or a portion of one) dune field. This resulted in a total dataset of 1116 image/label pairs.
The input data file mars_dunes.npz (about 1Gb) is available at https://drive.google.com/file/d/1mBn8mQ1O51EprTpXSNQtK-VSmF3wD7Z-/view?usp=sharing
Using Google Drive, you can create a shortcut for a shared file:
from google.colab import drive
drive.mount('/content/gdrive')
Mounted at /content/gdrive
import numpy as np
loaded = np.load('/content/gdrive/My Drive/mars_dunes.npz')
data = loaded['data']
labels = loaded['labels']
print(data.shape,labels.shape)
(1116, 256, 256) (1116, 256, 256)
The are 1116 two-dimensional training images with dimensions $256 \times 256$. We have only included image tiles that have labels, which excludes a large portion of images tiles. Below, we visualize a random sampling of image tiles to better understand our training dataset.
import matplotlib.pyplot as plt
import random
def plot_mars_image(data,title):
random.seed(2000)
fig, axs = plt.subplots(nrows=5, ncols=5, figsize=(20, 12),
subplot_kw={'xticks': [], 'yticks': []})
for ax in axs.flat:
pick = random.randint(0,data.shape[0]-1)
ax.imshow(data[pick,:,:],cmap='gray')
plt.tight_layout()
fig.suptitle(title,fontsize=20)
plt.subplots_adjust(top=0.9)
plt.show()
plot_mars_image(data,'Training Data: THEMIS Images Tiles (256 x 256 pixels)')
We can also look at the corresponding labels that will be used in the training and validation of our neural network. We visualize the labels corresponding to the same random sampling of dune fields below.
def plot_dune(data,title):
fig, axs = plt.subplots(nrows=5, ncols=5, figsize=(20, 12),
subplot_kw={'xticks': [], 'yticks': []})
random.seed(2000)
for ax in axs.flat:
pick = random.randint(0,data.shape[0]-1)
ax.imshow(data[pick,:,:],cmap='gray_r',interpolation='bilinear')
plt.tight_layout()
fig.suptitle(title,fontsize=20)
plt.subplots_adjust(top=0.9)
plt.show()
plot_dune(labels,'Training Labels: Mars Global Dune Database Polygons')
Notice that most image/label tiles only capture a portion of a dune field (some only a very small portion). Below we visualize the image data and labels together.
def plot_mars_dune(data,labels,title):
fig, axs = plt.subplots(nrows=5, ncols=5, figsize=(20, 12),
subplot_kw={'xticks': [], 'yticks': []})
random.seed(2000)
for ax in axs.flat:
pick = random.randint(0,data.shape[0]-1)
ax.imshow(data[pick,:,:],cmap='gray')
# overlay fault labels with alpha for transparency
ax.imshow(labels[pick,:,:],cmap='gnuplot',alpha=0.5)
plt.tight_layout()
fig.suptitle(title,fontsize=20)
plt.subplots_adjust(top=0.9)
plt.show()
plot_mars_dune(data,labels,'Training Data: THEMIS Images and Dune Field Polygons Plotted Together')
To prepare the data for training, we will first normalize each image by subtracting the mean value and dividing by the standard deviation, resulting in each image having a mean of 0 and a standard deviation of 1. A fourth dimension of 1 is added to indicate that the input dataset has one channel.
# normalize data
mean = data.mean(axis=(1,2))[:,np.newaxis,np.newaxis]
std = data.std(axis=(1,2))[:,np.newaxis,np.newaxis]
data_normal = (data-mean)/std
#add dimension of 1
data_normal = np.expand_dims(data_normal,axis=3)
data_normal.shape
(1116, 256, 256, 1)
We randomly split the data for training (80%) and validation (20%).
from sklearn.model_selection import train_test_split
# Split the data for training and validation
data_train,data_valid,labels_train,labels_valid = \
train_test_split(data_normal,labels,test_size=0.2,shuffle= True)
We are going to use a convolutional neural network (CNN) accelerated on GPUs.
When used Google Colab for this project, utilizing their GPU for accelerating the training of our CNN.
from tensorflow import config
# setup GPU card
gpus = config.list_physical_devices('GPU')
if gpus:
for gpu in gpus:
print(gpu)
# dynamic memory allocation
config.experimental.set_memory_growth(gpu,True)
PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')
Now we can design our Unet-based neural network.
from tensorflow import keras
from tensorflow.keras import layers
nb_filter = [16,32,64,128,256]
def Unet(nb_filter):
keras.backend.clear_session()
# can take arbitrary input size
image = keras.Input((None,None,1),name='input')
conv0 = layers.Conv2D(nb_filter[0], (3,3), padding='same')(image)
conv0 = layers.BatchNormalization()(conv0)
conv0 = layers.Activation('relu')(conv0)
conv0 = layers.Conv2D(nb_filter[0], (3,3), padding='same')(conv0)
conv0 = layers.BatchNormalization()(conv0)
conv0 = layers.Activation('relu')(conv0)
pool0 = layers.MaxPooling2D(pool_size=(2,2))(conv0)
conv1 = layers.Conv2D(nb_filter[1], (3,3), padding='same')(pool0)
conv1 = layers.BatchNormalization()(conv1)
conv1 = layers.Activation('relu')(conv1)
conv1 = layers.Conv2D(nb_filter[1], (3,3), padding='same')(conv1)
conv1 = layers.BatchNormalization()(conv1)
conv1 = layers.Activation('relu')(conv1)
pool1 = layers.MaxPooling2D(pool_size=(2,2))(conv1)
conv2 = layers.Conv2D(nb_filter[2], (3,3), padding='same')(pool1)
conv2 = layers.BatchNormalization()(conv2)
conv2 = layers.Activation('relu')(conv2)
conv2 = layers.Conv2D(nb_filter[2], (3,3), padding='same')(conv2)
conv2 = layers.BatchNormalization()(conv2)
conv2 = layers.Activation('relu')(conv2)
pool2 = layers.MaxPooling2D(pool_size=(2,2))(conv2)
conv3 = layers.Conv2D(nb_filter[3], (3,3), padding='same')(pool2)
conv3 = layers.BatchNormalization()(conv3)
conv3 = layers.Activation('relu')(conv3)
conv3 = layers.Conv2D(nb_filter[3], (3,3), padding='same')(conv3)
conv3 = layers.BatchNormalization()(conv3)
conv3 = layers.Activation('relu')(conv3)
pool3 = layers.MaxPooling2D(pool_size=(2,2))(conv3)
conv4 = layers.Conv2D(nb_filter[4], (3,3), padding='same')(pool3)
conv4 = layers.BatchNormalization()(conv4)
conv4 = layers.Activation('relu')(conv4)
conv4 = layers.Conv2D(nb_filter[4], (3,3), padding='same')(conv4)
conv4 = layers.BatchNormalization()(conv4)
conv4 = layers.Activation('relu')(conv4)
up5 = layers.concatenate([layers.UpSampling2D(size=(2,2))(conv4), conv3], axis=-1)
conv5 = layers.Conv2D(nb_filter[3], (3,3), padding='same')(up5)
conv5 = layers.BatchNormalization()(conv5)
conv5 = layers.Activation('relu')(conv5)
conv5 = layers.Conv2D(nb_filter[3], (3,3), padding='same')(conv5)
conv5 = layers.BatchNormalization()(conv5)
conv5 = layers.Activation('relu')(conv5)
up6 = layers.concatenate([layers.UpSampling2D(size=(2,2))(conv5), conv2], axis=-1)
conv6 = layers.Conv2D(nb_filter[2], (3,3), padding='same')(up6)
conv6 = layers.BatchNormalization()(conv6)
conv6 = layers.Activation('relu')(conv6)
conv6 = layers.Conv2D(nb_filter[2], (3,3), padding='same')(conv6)
conv6 = layers.BatchNormalization()(conv6)
conv6 = layers.Activation('relu')(conv6)
up7 = layers.concatenate([layers.UpSampling2D(size=(2,2))(conv6), conv1], axis=-1)
conv7 = layers.Conv2D(nb_filter[1], (3,3), padding='same')(up7)
conv7 = layers.BatchNormalization()(conv7)
conv7 = layers.Activation('relu')(conv7)
conv7 = layers.Conv2D(nb_filter[1], (3,3), padding='same')(conv7)
conv7 = layers.BatchNormalization()(conv7)
conv7 = layers.Activation('relu')(conv7)
up8 = layers.concatenate([layers.UpSampling2D(size=(2,2))(conv7), conv0], axis=-1)
conv8 = layers.Conv2D(nb_filter[0], (3,3), padding='same')(up8)
conv8 = layers.BatchNormalization()(conv8)
conv8 = layers.Activation('relu')(conv8)
conv8 = layers.Conv2D(nb_filter[0], (3,3), padding='same')(conv8)
conv8 = layers.BatchNormalization()(conv8)
conv8 = layers.Activation('relu')(conv8)
conv9 = layers.Conv2D(1, (1,1), activation='sigmoid')(conv8)
model = keras.Model(inputs=[image], outputs=[conv9])
return model
We take a look at the model constuction, below.
model=Unet(nb_filter)
model.summary()
Model: "model"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input (InputLayer) [(None, None, None, 0
__________________________________________________________________________________________________
conv2d (Conv2D) (None, None, None, 1 160 input[0][0]
__________________________________________________________________________________________________
batch_normalization (BatchNorma (None, None, None, 1 64 conv2d[0][0]
__________________________________________________________________________________________________
activation (Activation) (None, None, None, 1 0 batch_normalization[0][0]
__________________________________________________________________________________________________
conv2d_1 (Conv2D) (None, None, None, 1 2320 activation[0][0]
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, None, None, 1 64 conv2d_1[0][0]
__________________________________________________________________________________________________
activation_1 (Activation) (None, None, None, 1 0 batch_normalization_1[0][0]
__________________________________________________________________________________________________
max_pooling2d (MaxPooling2D) (None, None, None, 1 0 activation_1[0][0]
__________________________________________________________________________________________________
conv2d_2 (Conv2D) (None, None, None, 3 4640 max_pooling2d[0][0]
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, None, None, 3 128 conv2d_2[0][0]
__________________________________________________________________________________________________
activation_2 (Activation) (None, None, None, 3 0 batch_normalization_2[0][0]
__________________________________________________________________________________________________
conv2d_3 (Conv2D) (None, None, None, 3 9248 activation_2[0][0]
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, None, None, 3 128 conv2d_3[0][0]
__________________________________________________________________________________________________
activation_3 (Activation) (None, None, None, 3 0 batch_normalization_3[0][0]
__________________________________________________________________________________________________
max_pooling2d_1 (MaxPooling2D) (None, None, None, 3 0 activation_3[0][0]
__________________________________________________________________________________________________
conv2d_4 (Conv2D) (None, None, None, 6 18496 max_pooling2d_1[0][0]
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, None, None, 6 256 conv2d_4[0][0]
__________________________________________________________________________________________________
activation_4 (Activation) (None, None, None, 6 0 batch_normalization_4[0][0]
__________________________________________________________________________________________________
conv2d_5 (Conv2D) (None, None, None, 6 36928 activation_4[0][0]
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, None, None, 6 256 conv2d_5[0][0]
__________________________________________________________________________________________________
activation_5 (Activation) (None, None, None, 6 0 batch_normalization_5[0][0]
__________________________________________________________________________________________________
max_pooling2d_2 (MaxPooling2D) (None, None, None, 6 0 activation_5[0][0]
__________________________________________________________________________________________________
conv2d_6 (Conv2D) (None, None, None, 1 73856 max_pooling2d_2[0][0]
__________________________________________________________________________________________________
batch_normalization_6 (BatchNor (None, None, None, 1 512 conv2d_6[0][0]
__________________________________________________________________________________________________
activation_6 (Activation) (None, None, None, 1 0 batch_normalization_6[0][0]
__________________________________________________________________________________________________
conv2d_7 (Conv2D) (None, None, None, 1 147584 activation_6[0][0]
__________________________________________________________________________________________________
batch_normalization_7 (BatchNor (None, None, None, 1 512 conv2d_7[0][0]
__________________________________________________________________________________________________
activation_7 (Activation) (None, None, None, 1 0 batch_normalization_7[0][0]
__________________________________________________________________________________________________
max_pooling2d_3 (MaxPooling2D) (None, None, None, 1 0 activation_7[0][0]
__________________________________________________________________________________________________
conv2d_8 (Conv2D) (None, None, None, 2 295168 max_pooling2d_3[0][0]
__________________________________________________________________________________________________
batch_normalization_8 (BatchNor (None, None, None, 2 1024 conv2d_8[0][0]
__________________________________________________________________________________________________
activation_8 (Activation) (None, None, None, 2 0 batch_normalization_8[0][0]
__________________________________________________________________________________________________
conv2d_9 (Conv2D) (None, None, None, 2 590080 activation_8[0][0]
__________________________________________________________________________________________________
batch_normalization_9 (BatchNor (None, None, None, 2 1024 conv2d_9[0][0]
__________________________________________________________________________________________________
activation_9 (Activation) (None, None, None, 2 0 batch_normalization_9[0][0]
__________________________________________________________________________________________________
up_sampling2d (UpSampling2D) (None, None, None, 2 0 activation_9[0][0]
__________________________________________________________________________________________________
concatenate (Concatenate) (None, None, None, 3 0 up_sampling2d[0][0]
activation_7[0][0]
__________________________________________________________________________________________________
conv2d_10 (Conv2D) (None, None, None, 1 442496 concatenate[0][0]
__________________________________________________________________________________________________
batch_normalization_10 (BatchNo (None, None, None, 1 512 conv2d_10[0][0]
__________________________________________________________________________________________________
activation_10 (Activation) (None, None, None, 1 0 batch_normalization_10[0][0]
__________________________________________________________________________________________________
conv2d_11 (Conv2D) (None, None, None, 1 147584 activation_10[0][0]
__________________________________________________________________________________________________
batch_normalization_11 (BatchNo (None, None, None, 1 512 conv2d_11[0][0]
__________________________________________________________________________________________________
activation_11 (Activation) (None, None, None, 1 0 batch_normalization_11[0][0]
__________________________________________________________________________________________________
up_sampling2d_1 (UpSampling2D) (None, None, None, 1 0 activation_11[0][0]
__________________________________________________________________________________________________
concatenate_1 (Concatenate) (None, None, None, 1 0 up_sampling2d_1[0][0]
activation_5[0][0]
__________________________________________________________________________________________________
conv2d_12 (Conv2D) (None, None, None, 6 110656 concatenate_1[0][0]
__________________________________________________________________________________________________
batch_normalization_12 (BatchNo (None, None, None, 6 256 conv2d_12[0][0]
__________________________________________________________________________________________________
activation_12 (Activation) (None, None, None, 6 0 batch_normalization_12[0][0]
__________________________________________________________________________________________________
conv2d_13 (Conv2D) (None, None, None, 6 36928 activation_12[0][0]
__________________________________________________________________________________________________
batch_normalization_13 (BatchNo (None, None, None, 6 256 conv2d_13[0][0]
__________________________________________________________________________________________________
activation_13 (Activation) (None, None, None, 6 0 batch_normalization_13[0][0]
__________________________________________________________________________________________________
up_sampling2d_2 (UpSampling2D) (None, None, None, 6 0 activation_13[0][0]
__________________________________________________________________________________________________
concatenate_2 (Concatenate) (None, None, None, 9 0 up_sampling2d_2[0][0]
activation_3[0][0]
__________________________________________________________________________________________________
conv2d_14 (Conv2D) (None, None, None, 3 27680 concatenate_2[0][0]
__________________________________________________________________________________________________
batch_normalization_14 (BatchNo (None, None, None, 3 128 conv2d_14[0][0]
__________________________________________________________________________________________________
activation_14 (Activation) (None, None, None, 3 0 batch_normalization_14[0][0]
__________________________________________________________________________________________________
conv2d_15 (Conv2D) (None, None, None, 3 9248 activation_14[0][0]
__________________________________________________________________________________________________
batch_normalization_15 (BatchNo (None, None, None, 3 128 conv2d_15[0][0]
__________________________________________________________________________________________________
activation_15 (Activation) (None, None, None, 3 0 batch_normalization_15[0][0]
__________________________________________________________________________________________________
up_sampling2d_3 (UpSampling2D) (None, None, None, 3 0 activation_15[0][0]
__________________________________________________________________________________________________
concatenate_3 (Concatenate) (None, None, None, 4 0 up_sampling2d_3[0][0]
activation_1[0][0]
__________________________________________________________________________________________________
conv2d_16 (Conv2D) (None, None, None, 1 6928 concatenate_3[0][0]
__________________________________________________________________________________________________
batch_normalization_16 (BatchNo (None, None, None, 1 64 conv2d_16[0][0]
__________________________________________________________________________________________________
activation_16 (Activation) (None, None, None, 1 0 batch_normalization_16[0][0]
__________________________________________________________________________________________________
conv2d_17 (Conv2D) (None, None, None, 1 2320 activation_16[0][0]
__________________________________________________________________________________________________
batch_normalization_17 (BatchNo (None, None, None, 1 64 conv2d_17[0][0]
__________________________________________________________________________________________________
activation_17 (Activation) (None, None, None, 1 0 batch_normalization_17[0][0]
__________________________________________________________________________________________________
conv2d_18 (Conv2D) (None, None, None, 1 17 activation_17[0][0]
==================================================================================================
Total params: 1,968,225
Trainable params: 1,965,281
Non-trainable params: 2,944
__________________________________________________________________________________________________
Our model has 1,968,225 total parameters.
keras.utils.plot_model(model,show_shapes=True)
Now we are ready to train our model. As before, we will use orginal size (1116 image tiles) of the dataset, cross-entropy as the loss function (applied pixel-by-pixel in the image), and Adam as the optimization algorithm.
model0=Unet(nb_filter)
model0.compile(loss='binary_crossentropy',
optimizer='adam',
metrics=['accuracy'])
# Fit the model
batch_size = 40
epochs = 50
fit0 = model0.fit(data_train,labels_train,
validation_data=(data_valid,labels_valid),
epochs=epochs,
batch_size=batch_size)
Epoch 1/50 23/23 [==============================] - 53s 623ms/step - loss: 0.4721 - accuracy: 0.8088 - val_loss: 7.3836 - val_accuracy: 0.1154 Epoch 2/50 23/23 [==============================] - 10s 431ms/step - loss: 0.2846 - accuracy: 0.9144 - val_loss: 4.4437 - val_accuracy: 0.2130 Epoch 3/50 23/23 [==============================] - 10s 434ms/step - loss: 0.2620 - accuracy: 0.9112 - val_loss: 0.7318 - val_accuracy: 0.5912 Epoch 4/50 23/23 [==============================] - 10s 440ms/step - loss: 0.2530 - accuracy: 0.9107 - val_loss: 0.3456 - val_accuracy: 0.8982 Epoch 5/50 23/23 [==============================] - 10s 444ms/step - loss: 0.2320 - accuracy: 0.9180 - val_loss: 0.3209 - val_accuracy: 0.8949 Epoch 6/50 23/23 [==============================] - 10s 449ms/step - loss: 0.2108 - accuracy: 0.9268 - val_loss: 0.3087 - val_accuracy: 0.8939 Epoch 7/50 23/23 [==============================] - 10s 456ms/step - loss: 0.2090 - accuracy: 0.9261 - val_loss: 0.2653 - val_accuracy: 0.9021 Epoch 8/50 23/23 [==============================] - 11s 464ms/step - loss: 0.2219 - accuracy: 0.9183 - val_loss: 0.2815 - val_accuracy: 0.8977 Epoch 9/50 23/23 [==============================] - 11s 471ms/step - loss: 0.2169 - accuracy: 0.9202 - val_loss: 0.2975 - val_accuracy: 0.9016 Epoch 10/50 23/23 [==============================] - 11s 469ms/step - loss: 0.2042 - accuracy: 0.9265 - val_loss: 0.2469 - val_accuracy: 0.9181 Epoch 11/50 23/23 [==============================] - 11s 463ms/step - loss: 0.1963 - accuracy: 0.9281 - val_loss: 0.2424 - val_accuracy: 0.9241 Epoch 12/50 23/23 [==============================] - 11s 458ms/step - loss: 0.1910 - accuracy: 0.9305 - val_loss: 0.2330 - val_accuracy: 0.9270 Epoch 13/50 23/23 [==============================] - 10s 457ms/step - loss: 0.1887 - accuracy: 0.9304 - val_loss: 0.2060 - val_accuracy: 0.9313 Epoch 14/50 23/23 [==============================] - 10s 457ms/step - loss: 0.1842 - accuracy: 0.9321 - val_loss: 0.2191 - val_accuracy: 0.9264 Epoch 15/50 23/23 [==============================] - 11s 459ms/step - loss: 0.1758 - accuracy: 0.9345 - val_loss: 0.2099 - val_accuracy: 0.9320 Epoch 16/50 23/23 [==============================] - 11s 462ms/step - loss: 0.1666 - accuracy: 0.9405 - val_loss: 0.2333 - val_accuracy: 0.9148 Epoch 17/50 23/23 [==============================] - 11s 464ms/step - loss: 0.1900 - accuracy: 0.9280 - val_loss: 0.1953 - val_accuracy: 0.9312 Epoch 18/50 23/23 [==============================] - 11s 464ms/step - loss: 0.1764 - accuracy: 0.9337 - val_loss: 0.1854 - val_accuracy: 0.9367 Epoch 19/50 23/23 [==============================] - 11s 464ms/step - loss: 0.1644 - accuracy: 0.9396 - val_loss: 0.1762 - val_accuracy: 0.9377 Epoch 20/50 23/23 [==============================] - 11s 463ms/step - loss: 0.1605 - accuracy: 0.9398 - val_loss: 0.1978 - val_accuracy: 0.9284 Epoch 21/50 23/23 [==============================] - 11s 463ms/step - loss: 0.1758 - accuracy: 0.9347 - val_loss: 0.2223 - val_accuracy: 0.8974 Epoch 22/50 23/23 [==============================] - 11s 462ms/step - loss: 0.1669 - accuracy: 0.9362 - val_loss: 0.2005 - val_accuracy: 0.9073 Epoch 23/50 23/23 [==============================] - 11s 463ms/step - loss: 0.1533 - accuracy: 0.9424 - val_loss: 0.1647 - val_accuracy: 0.9368 Epoch 24/50 23/23 [==============================] - 11s 463ms/step - loss: 0.1618 - accuracy: 0.9397 - val_loss: 0.1756 - val_accuracy: 0.9377 Epoch 25/50 23/23 [==============================] - 11s 462ms/step - loss: 0.1500 - accuracy: 0.9431 - val_loss: 0.1904 - val_accuracy: 0.9256 Epoch 26/50 23/23 [==============================] - 11s 459ms/step - loss: 0.1535 - accuracy: 0.9397 - val_loss: 0.2046 - val_accuracy: 0.9154 Epoch 27/50 23/23 [==============================] - 11s 460ms/step - loss: 0.1581 - accuracy: 0.9383 - val_loss: 0.2578 - val_accuracy: 0.9117 Epoch 28/50 23/23 [==============================] - 11s 462ms/step - loss: 0.1541 - accuracy: 0.9404 - val_loss: 0.1971 - val_accuracy: 0.9258 Epoch 29/50 23/23 [==============================] - 11s 463ms/step - loss: 0.1375 - accuracy: 0.9487 - val_loss: 0.1740 - val_accuracy: 0.9342 Epoch 30/50 23/23 [==============================] - 11s 465ms/step - loss: 0.1278 - accuracy: 0.9523 - val_loss: 0.1828 - val_accuracy: 0.9293 Epoch 31/50 23/23 [==============================] - 11s 464ms/step - loss: 0.1312 - accuracy: 0.9506 - val_loss: 0.2019 - val_accuracy: 0.9131 Epoch 32/50 23/23 [==============================] - 11s 464ms/step - loss: 0.1218 - accuracy: 0.9532 - val_loss: 0.1753 - val_accuracy: 0.9377 Epoch 33/50 23/23 [==============================] - 11s 465ms/step - loss: 0.1266 - accuracy: 0.9514 - val_loss: 0.2507 - val_accuracy: 0.9304 Epoch 34/50 23/23 [==============================] - 11s 463ms/step - loss: 0.1257 - accuracy: 0.9512 - val_loss: 0.1866 - val_accuracy: 0.9427 Epoch 35/50 23/23 [==============================] - 11s 462ms/step - loss: 0.1119 - accuracy: 0.9567 - val_loss: 0.1574 - val_accuracy: 0.9450 Epoch 36/50 23/23 [==============================] - 11s 462ms/step - loss: 0.1104 - accuracy: 0.9575 - val_loss: 0.2004 - val_accuracy: 0.9232 Epoch 37/50 23/23 [==============================] - 11s 461ms/step - loss: 0.1116 - accuracy: 0.9569 - val_loss: 0.1661 - val_accuracy: 0.9391 Epoch 38/50 23/23 [==============================] - 11s 463ms/step - loss: 0.1060 - accuracy: 0.9588 - val_loss: 0.1910 - val_accuracy: 0.9337 Epoch 39/50 23/23 [==============================] - 11s 461ms/step - loss: 0.0978 - accuracy: 0.9628 - val_loss: 0.1741 - val_accuracy: 0.9384 Epoch 40/50 23/23 [==============================] - 11s 461ms/step - loss: 0.0976 - accuracy: 0.9627 - val_loss: 0.1844 - val_accuracy: 0.9349 Epoch 41/50 23/23 [==============================] - 11s 462ms/step - loss: 0.1014 - accuracy: 0.9630 - val_loss: 0.2121 - val_accuracy: 0.9263 Epoch 42/50 23/23 [==============================] - 11s 461ms/step - loss: 0.0990 - accuracy: 0.9624 - val_loss: 0.2298 - val_accuracy: 0.9148 Epoch 43/50 23/23 [==============================] - 11s 463ms/step - loss: 0.0961 - accuracy: 0.9625 - val_loss: 0.1785 - val_accuracy: 0.9436 Epoch 44/50 23/23 [==============================] - 11s 463ms/step - loss: 0.0797 - accuracy: 0.9704 - val_loss: 0.1950 - val_accuracy: 0.9421 Epoch 45/50 23/23 [==============================] - 11s 463ms/step - loss: 0.0750 - accuracy: 0.9721 - val_loss: 0.1880 - val_accuracy: 0.9365 Epoch 46/50 23/23 [==============================] - 11s 464ms/step - loss: 0.0777 - accuracy: 0.9704 - val_loss: 0.2515 - val_accuracy: 0.9345 Epoch 47/50 23/23 [==============================] - 11s 462ms/step - loss: 0.0793 - accuracy: 0.9700 - val_loss: 0.2445 - val_accuracy: 0.9059 Epoch 48/50 23/23 [==============================] - 11s 463ms/step - loss: 0.0759 - accuracy: 0.9714 - val_loss: 0.1914 - val_accuracy: 0.9247 Epoch 49/50 23/23 [==============================] - 11s 463ms/step - loss: 0.0648 - accuracy: 0.9755 - val_loss: 0.1710 - val_accuracy: 0.9421 Epoch 50/50 23/23 [==============================] - 11s 464ms/step - loss: 0.0707 - accuracy: 0.9732 - val_loss: 0.2204 - val_accuracy: 0.9404
def plot_history(history):
fig = plt.figure(figsize=(10,6))
plt.plot(history['accuracy'],label='training')
plt.plot(history['val_accuracy'],label='validation')
plt.ylabel('Accuracy',fontsize=20)
plt.xlabel('Epoch',fontsize=20)
plt.legend(fontsize=20)
plt.show()
# summarize history for loss
fig = plt.figure(figsize=(10,6))
plt.plot(history['loss'],label='training')
plt.plot(history['val_loss'],label='validation')
plt.ylabel('Loss',fontsize=20)
plt.xlabel('Epoch',fontsize=20)
plt.legend(fontsize=20)
plt.show()
plot_history(fit0.history)
More data is always (usually?) better. We now extend the dataset to be four times larger than our orginal one by flipping and rotating the dataset.
# flipping data
f_data = np.flip(data,axis=1)
f_labels = np.flip(labels,axis=1)
# rotate data
r_data = np.flip(f_data,axis=0)
r_labels = np.flip(f_labels,axis=0)
# flipping again
h_data = np.flip(data,axis=0)
h_labels = np.flip(labels,axis=0)
t_data = np.concatenate([data, r_data, h_data, f_data], axis=0)
t_labels = np.concatenate([labels, r_labels, h_labels, f_labels], axis=0)
print(data.shape,labels.shape)
print(t_data.shape,t_labels.shape)
# normalize data
mean = t_data.mean(axis=(1,2))[:,np.newaxis,np.newaxis]
std = t_data.std(axis=(1,2))[:,np.newaxis,np.newaxis]
t_data_normal = (t_data-mean)/std
#add dimension of 1
t_data = np.expand_dims(t_data_normal,axis=3)
t_data.shape
(1116, 256, 256) (1116, 256, 256) (4464, 256, 256) (4464, 256, 256)
(4464, 256, 256, 1)
# Split the data for training and validation
t_data_train,t_data_valid,t_labels_train,t_labels_valid = \
train_test_split(t_data,t_labels,test_size=0.2,shuffle= True)
t_data_train.shape
(3571, 256, 256, 1)
model=Unet(nb_filter)
model.compile(loss='binary_crossentropy',
optimizer='adam',
metrics=['accuracy'])
# Fit the model
batch_size = 40
epochs = 50
fit = model.fit(t_data_train,t_labels_train,
validation_data=(t_data_valid,t_labels_valid),
epochs=epochs,
batch_size=batch_size)
Epoch 1/50 90/90 [==============================] - 26s 264ms/step - loss: 0.5931 - accuracy: 0.7568 - val_loss: 0.6724 - val_accuracy: 0.5428 Epoch 2/50 90/90 [==============================] - 22s 245ms/step - loss: 0.3411 - accuracy: 0.9216 - val_loss: 0.3834 - val_accuracy: 0.8745 Epoch 3/50 90/90 [==============================] - 22s 245ms/step - loss: 0.2761 - accuracy: 0.9210 - val_loss: 0.2695 - val_accuracy: 0.9114 Epoch 4/50 90/90 [==============================] - 22s 245ms/step - loss: 0.2305 - accuracy: 0.9281 - val_loss: 0.2808 - val_accuracy: 0.9069 Epoch 5/50 90/90 [==============================] - 22s 245ms/step - loss: 0.2135 - accuracy: 0.9261 - val_loss: 0.2335 - val_accuracy: 0.9137 Epoch 6/50 90/90 [==============================] - 22s 245ms/step - loss: 0.1951 - accuracy: 0.9320 - val_loss: 0.2940 - val_accuracy: 0.8656 Epoch 7/50 90/90 [==============================] - 22s 245ms/step - loss: 0.2034 - accuracy: 0.9256 - val_loss: 0.2175 - val_accuracy: 0.9216 Epoch 8/50 90/90 [==============================] - 22s 245ms/step - loss: 0.1804 - accuracy: 0.9347 - val_loss: 0.1823 - val_accuracy: 0.9363 Epoch 9/50 90/90 [==============================] - 22s 245ms/step - loss: 0.1702 - accuracy: 0.9380 - val_loss: 0.1855 - val_accuracy: 0.9324 Epoch 10/50 90/90 [==============================] - 22s 245ms/step - loss: 0.1686 - accuracy: 0.9387 - val_loss: 0.1997 - val_accuracy: 0.9312 Epoch 11/50 90/90 [==============================] - 22s 245ms/step - loss: 0.1665 - accuracy: 0.9381 - val_loss: 0.2422 - val_accuracy: 0.9018 Epoch 12/50 90/90 [==============================] - 22s 245ms/step - loss: 0.1636 - accuracy: 0.9401 - val_loss: 0.1609 - val_accuracy: 0.9391 Epoch 13/50 90/90 [==============================] - 22s 245ms/step - loss: 0.1442 - accuracy: 0.9473 - val_loss: 0.1566 - val_accuracy: 0.9423 Epoch 14/50 90/90 [==============================] - 22s 245ms/step - loss: 0.1421 - accuracy: 0.9465 - val_loss: 0.1738 - val_accuracy: 0.9307 Epoch 15/50 90/90 [==============================] - 22s 245ms/step - loss: 0.1465 - accuracy: 0.9446 - val_loss: 0.1778 - val_accuracy: 0.9372 Epoch 16/50 90/90 [==============================] - 22s 245ms/step - loss: 0.1495 - accuracy: 0.9434 - val_loss: 0.1445 - val_accuracy: 0.9455 Epoch 17/50 90/90 [==============================] - 22s 245ms/step - loss: 0.1349 - accuracy: 0.9500 - val_loss: 0.1468 - val_accuracy: 0.9453 Epoch 18/50 90/90 [==============================] - 22s 245ms/step - loss: 0.1306 - accuracy: 0.9507 - val_loss: 0.1679 - val_accuracy: 0.9374 Epoch 19/50 90/90 [==============================] - 22s 245ms/step - loss: 0.1235 - accuracy: 0.9528 - val_loss: 0.1404 - val_accuracy: 0.9447 Epoch 20/50 90/90 [==============================] - 22s 245ms/step - loss: 0.1193 - accuracy: 0.9554 - val_loss: 0.1403 - val_accuracy: 0.9471 Epoch 21/50 90/90 [==============================] - 22s 245ms/step - loss: 0.1087 - accuracy: 0.9588 - val_loss: 0.1601 - val_accuracy: 0.9369 Epoch 22/50 90/90 [==============================] - 22s 245ms/step - loss: 0.1055 - accuracy: 0.9602 - val_loss: 0.1170 - val_accuracy: 0.9553 Epoch 23/50 90/90 [==============================] - 22s 245ms/step - loss: 0.1067 - accuracy: 0.9596 - val_loss: 0.1468 - val_accuracy: 0.9419 Epoch 24/50 90/90 [==============================] - 22s 245ms/step - loss: 0.1062 - accuracy: 0.9601 - val_loss: 0.1505 - val_accuracy: 0.9400 Epoch 25/50 90/90 [==============================] - 22s 244ms/step - loss: 0.0972 - accuracy: 0.9632 - val_loss: 0.1921 - val_accuracy: 0.9332 Epoch 26/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0851 - accuracy: 0.9676 - val_loss: 0.1353 - val_accuracy: 0.9470 Epoch 27/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0793 - accuracy: 0.9701 - val_loss: 0.1012 - val_accuracy: 0.9622 Epoch 28/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0797 - accuracy: 0.9695 - val_loss: 0.1006 - val_accuracy: 0.9610 Epoch 29/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0688 - accuracy: 0.9735 - val_loss: 0.0966 - val_accuracy: 0.9640 Epoch 30/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0653 - accuracy: 0.9749 - val_loss: 0.1128 - val_accuracy: 0.9589 Epoch 31/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0610 - accuracy: 0.9764 - val_loss: 0.0983 - val_accuracy: 0.9649 Epoch 32/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0536 - accuracy: 0.9792 - val_loss: 0.0918 - val_accuracy: 0.9652 Epoch 33/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0702 - accuracy: 0.9726 - val_loss: 0.0827 - val_accuracy: 0.9698 Epoch 34/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0502 - accuracy: 0.9803 - val_loss: 0.0881 - val_accuracy: 0.9657 Epoch 35/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0442 - accuracy: 0.9826 - val_loss: 0.0913 - val_accuracy: 0.9693 Epoch 36/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0385 - accuracy: 0.9846 - val_loss: 0.5002 - val_accuracy: 0.9246 Epoch 37/50 90/90 [==============================] - 22s 244ms/step - loss: 0.0472 - accuracy: 0.9814 - val_loss: 0.0915 - val_accuracy: 0.9704 Epoch 38/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0377 - accuracy: 0.9849 - val_loss: 0.0700 - val_accuracy: 0.9756 Epoch 39/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0321 - accuracy: 0.9871 - val_loss: 0.0786 - val_accuracy: 0.9734 Epoch 40/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0337 - accuracy: 0.9865 - val_loss: 0.0767 - val_accuracy: 0.9744 Epoch 41/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0304 - accuracy: 0.9878 - val_loss: 0.0775 - val_accuracy: 0.9760 Epoch 42/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0288 - accuracy: 0.9883 - val_loss: 0.0680 - val_accuracy: 0.9778 Epoch 43/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0264 - accuracy: 0.9893 - val_loss: 0.0666 - val_accuracy: 0.9800 Epoch 44/50 90/90 [==============================] - 22s 244ms/step - loss: 0.0243 - accuracy: 0.9902 - val_loss: 0.0788 - val_accuracy: 0.9776 Epoch 45/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0272 - accuracy: 0.9890 - val_loss: 0.0701 - val_accuracy: 0.9796 Epoch 46/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0228 - accuracy: 0.9907 - val_loss: 0.0746 - val_accuracy: 0.9779 Epoch 47/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0367 - accuracy: 0.9854 - val_loss: 0.2604 - val_accuracy: 0.9288 Epoch 48/50 90/90 [==============================] - 22s 244ms/step - loss: 0.0604 - accuracy: 0.9763 - val_loss: 0.0811 - val_accuracy: 0.9746 Epoch 49/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0279 - accuracy: 0.9888 - val_loss: 0.0641 - val_accuracy: 0.9801 Epoch 50/50 90/90 [==============================] - 22s 244ms/step - loss: 0.0197 - accuracy: 0.9920 - val_loss: 0.0618 - val_accuracy: 0.9825
plot_history(fit.history)
Since it is evident that the number of background shapes in any given detection image is usually more than the number of dune class instances, training a neural network might make it biased to outputting background class more compared to dune class and it may affects the performance of a neural net. The problem of class imbalance can be solved by adding more instances of the less dominant class in training data.

import tensorflow as tf
def _to_tensor(x, dtype):
x = tf.convert_to_tensor(x)
if x.dtype != dtype:
x = tf.cast(x, dtype)
return x
def cross_entropy_balanced(y_true, y_pred):
from keras import backend as K
_epsilon = _to_tensor(K.epsilon(), y_pred.dtype.base_dtype)
y_pred = tf.clip_by_value(y_pred, _epsilon, 1 - _epsilon)
y_pred = tf.math.log(y_pred/ (1 - y_pred))
y_true = tf.cast(y_true, tf.float32)
count_neg = tf.reduce_sum(1. - y_true)
count_pos = tf.reduce_sum(y_true)
beta = count_neg / (count_neg + count_pos)
pos_weight = beta / (1 - beta)
cost = tf.nn.weighted_cross_entropy_with_logits(logits=y_pred, labels=y_true, pos_weight=pos_weight)
cost = tf.reduce_mean(cost * (1 - beta))
return tf.where(tf.equal(count_pos, 0.0), 0.0, cost)
model1=Unet(nb_filter)
model1.compile(loss=cross_entropy_balanced,
#optimizer='adam',
optimizer=keras.optimizers.Adam(0.001),
metrics=['accuracy'])
# Fit the model
batch_size = 40
epochs = 50
fit = model1.fit(t_data_train,t_labels_train,
validation_data=(t_data_valid,t_labels_valid),
epochs=epochs,
batch_size=batch_size)
Epoch 1/50 90/90 [==============================] - 25s 253ms/step - loss: 0.0998 - accuracy: 0.7043 - val_loss: 0.1735 - val_accuracy: 0.7205 Epoch 2/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0763 - accuracy: 0.8120 - val_loss: 0.1947 - val_accuracy: 0.8844 Epoch 3/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0702 - accuracy: 0.8483 - val_loss: 0.1151 - val_accuracy: 0.9137 Epoch 4/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0697 - accuracy: 0.8491 - val_loss: 0.1371 - val_accuracy: 0.9171 Epoch 5/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0635 - accuracy: 0.8593 - val_loss: 0.0714 - val_accuracy: 0.8998 Epoch 6/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0613 - accuracy: 0.8663 - val_loss: 0.0761 - val_accuracy: 0.8762 Epoch 7/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0606 - accuracy: 0.8754 - val_loss: 0.0793 - val_accuracy: 0.9165 Epoch 8/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0614 - accuracy: 0.8732 - val_loss: 0.0662 - val_accuracy: 0.8716 Epoch 9/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0591 - accuracy: 0.8763 - val_loss: 0.0700 - val_accuracy: 0.8402 Epoch 10/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0563 - accuracy: 0.8840 - val_loss: 0.0614 - val_accuracy: 0.8721 Epoch 11/50 90/90 [==============================] - 22s 246ms/step - loss: 0.0513 - accuracy: 0.8865 - val_loss: 0.1234 - val_accuracy: 0.9338 Epoch 12/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0537 - accuracy: 0.8871 - val_loss: 0.0608 - val_accuracy: 0.8621 Epoch 13/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0452 - accuracy: 0.9001 - val_loss: 0.0575 - val_accuracy: 0.8834 Epoch 14/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0470 - accuracy: 0.8939 - val_loss: 0.0500 - val_accuracy: 0.8961 Epoch 15/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0399 - accuracy: 0.9123 - val_loss: 0.0537 - val_accuracy: 0.8887 Epoch 16/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0421 - accuracy: 0.9065 - val_loss: 0.0533 - val_accuracy: 0.9206 Epoch 17/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0371 - accuracy: 0.9208 - val_loss: 0.0570 - val_accuracy: 0.8810 Epoch 18/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0348 - accuracy: 0.9251 - val_loss: 0.0609 - val_accuracy: 0.9353 Epoch 19/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0324 - accuracy: 0.9241 - val_loss: 0.0498 - val_accuracy: 0.9144 Epoch 20/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0382 - accuracy: 0.9076 - val_loss: 0.0528 - val_accuracy: 0.8750 Epoch 21/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0281 - accuracy: 0.9329 - val_loss: 0.0421 - val_accuracy: 0.9241 Epoch 22/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0281 - accuracy: 0.9326 - val_loss: 0.0518 - val_accuracy: 0.9464 Epoch 23/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0273 - accuracy: 0.9382 - val_loss: 0.0380 - val_accuracy: 0.9411 Epoch 24/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0218 - accuracy: 0.9501 - val_loss: 0.0421 - val_accuracy: 0.9395 Epoch 25/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0214 - accuracy: 0.9515 - val_loss: 0.0854 - val_accuracy: 0.9502 Epoch 26/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0224 - accuracy: 0.9512 - val_loss: 0.0402 - val_accuracy: 0.9225 Epoch 27/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0190 - accuracy: 0.9560 - val_loss: 0.0370 - val_accuracy: 0.9254 Epoch 28/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0179 - accuracy: 0.9569 - val_loss: 0.0453 - val_accuracy: 0.9517 Epoch 29/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0175 - accuracy: 0.9592 - val_loss: 0.0505 - val_accuracy: 0.9380 Epoch 30/50 90/90 [==============================] - 22s 246ms/step - loss: 0.0173 - accuracy: 0.9583 - val_loss: 0.0332 - val_accuracy: 0.9512 Epoch 31/50 90/90 [==============================] - 22s 246ms/step - loss: 0.0154 - accuracy: 0.9649 - val_loss: 0.0401 - val_accuracy: 0.9382 Epoch 32/50 90/90 [==============================] - 22s 246ms/step - loss: 0.0189 - accuracy: 0.9567 - val_loss: 0.0353 - val_accuracy: 0.9576 Epoch 33/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0155 - accuracy: 0.9645 - val_loss: 0.0327 - val_accuracy: 0.9364 Epoch 34/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0139 - accuracy: 0.9657 - val_loss: 0.0320 - val_accuracy: 0.9616 Epoch 35/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0113 - accuracy: 0.9723 - val_loss: 0.0530 - val_accuracy: 0.9676 Epoch 36/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0115 - accuracy: 0.9717 - val_loss: 0.0304 - val_accuracy: 0.9632 Epoch 37/50 90/90 [==============================] - 22s 245ms/step - loss: 0.0110 - accuracy: 0.9734 - val_loss: 0.0314 - val_accuracy: 0.9475 Epoch 38/50 90/90 [==============================] - 22s 246ms/step - loss: 0.0107 - accuracy: 0.9738 - val_loss: 0.0418 - val_accuracy: 0.9702 Epoch 39/50 90/90 [==============================] - 22s 246ms/step - loss: 0.0123 - accuracy: 0.9716 - val_loss: 0.1152 - val_accuracy: 0.9506 Epoch 40/50 90/90 [==============================] - 22s 246ms/step - loss: 0.0186 - accuracy: 0.9591 - val_loss: 0.0351 - val_accuracy: 0.9585 Epoch 41/50 90/90 [==============================] - 22s 246ms/step - loss: 0.0108 - accuracy: 0.9726 - val_loss: 0.0338 - val_accuracy: 0.9697 Epoch 42/50 90/90 [==============================] - 22s 246ms/step - loss: 0.0087 - accuracy: 0.9792 - val_loss: 0.0348 - val_accuracy: 0.9696 Epoch 43/50 90/90 [==============================] - 22s 246ms/step - loss: 0.0075 - accuracy: 0.9819 - val_loss: 0.0328 - val_accuracy: 0.9671 Epoch 44/50 90/90 [==============================] - 22s 246ms/step - loss: 0.0084 - accuracy: 0.9792 - val_loss: 0.0342 - val_accuracy: 0.9681 Epoch 45/50 90/90 [==============================] - 22s 246ms/step - loss: 0.0068 - accuracy: 0.9838 - val_loss: 0.0400 - val_accuracy: 0.9749 Epoch 46/50 90/90 [==============================] - 22s 246ms/step - loss: 0.0063 - accuracy: 0.9848 - val_loss: 0.0372 - val_accuracy: 0.9719 Epoch 47/50 90/90 [==============================] - 22s 246ms/step - loss: 0.0091 - accuracy: 0.9786 - val_loss: 0.0329 - val_accuracy: 0.9651 Epoch 48/50 90/90 [==============================] - 22s 246ms/step - loss: 0.0107 - accuracy: 0.9743 - val_loss: 0.0292 - val_accuracy: 0.9672 Epoch 49/50 90/90 [==============================] - 22s 246ms/step - loss: 0.0086 - accuracy: 0.9798 - val_loss: 0.0379 - val_accuracy: 0.9766 Epoch 50/50 90/90 [==============================] - 22s 246ms/step - loss: 0.0062 - accuracy: 0.9849 - val_loss: 0.0348 - val_accuracy: 0.9731
plot_history(fit.history)
We save our model termed "deepdunes_model.h5", so that it can be reload throught the Google driver for further testing.
#Save model
keras.models.save_model(model,'deepdunes_model.h5',save_format='h5')
# #Load the model back with model
# model = keras.models.load_model('deepdunes_model.h5')
prediction = model.predict(t_data_valid)
# thresholding probability
labels_pred = prediction
labels_pred[prediction>=0.5]=1
labels_pred[prediction<0.5]=0
plot_dune(t_data_valid[:,:,:,0],'Example Validation Data')
plot_dune(t_labels_valid,'Example Validation Labels')
plot_dune(labels_pred[:,:,:,0],'Predicted Dune Field Labels')
plot_mars_dune(t_data_valid[:,:,:,0],labels_pred[:,:,:,0],'Predicted Labels')
We would like to know if there are any unmapped dune fields on Mars. To find potential candidates we can apply our model to the full unlabeled dataset (~76,000) 256 x 256 image tiles. We had to divide these into 38 files containing 2,000 tiles per file due to RAM and disk limitations. A shortcut to the image files can be found here. The model 'deepdunes_model.hf' can also be downloaded (or a shortcut can be made) here
import keras
import numpy as np
#Load model
model = keras.models.load_model('/content/gdrive/My Drive/deepdunes_model.h5')
#function for loading in each 2000 image .npz (38 total files)
n = np.arange(37)
file_path_list = []
for i in range(len(n)):
filename = '/content/gdrive/My Drive/mars_dunes_full_ds/mars_dunes_full_ds_'+str(i)+'.npz'
file_path_list.append(filename)
We now use our model to 'detect' pixels that may correspond to unmapped dune fields. This is only a preliminary result and needs more work!
import numpy as np
predicted_labels_list = []
predicted_data_list = []
for i in np.arange(len(file_path_list)):
print(file_path_list[i])
loaded = np.load(file_path_list[i], 'r')
testing_data = loaded['data']
#normalize data
mean = testing_data.mean(axis=(1,2))[:,np.newaxis,np.newaxis]
std = testing_data.std(axis=(1,2))[:,np.newaxis,np.newaxis]
testing_data_normal = (testing_data-mean)/std
#add dimension of 1
testing_data = np.expand_dims(testing_data,axis=3)
testing_data.shape
prediction = model.predict(testing_data)
# thresholding probability
labels_pred = prediction
labels_pred[prediction>=0.5]=1
labels_pred[prediction<0.5]=0
plot_mars_dune(testing_data[:,:,:,0],labels_pred[:,:,:,0],'Possible Dune Fields')
#clear memory/disk so the next file can be read in. See how to do this?
del loaded
del testing_data
keras.backend.clear_session()
/content/gdrive/My Drive/mars_dunes_full_ds/mars_dunes_full_ds_0.npz
/content/gdrive/My Drive/mars_dunes_full_ds/mars_dunes_full_ds_1.npz
/content/gdrive/My Drive/mars_dunes_full_ds/mars_dunes_full_ds_2.npz
/content/gdrive/My Drive/mars_dunes_full_ds/mars_dunes_full_ds_3.npz
/content/gdrive/My Drive/mars_dunes_full_ds/mars_dunes_full_ds_4.npz
Our training dataset included 454 hand-mapped dune fields. Using our model, we identify additional regions which may correspond to dune fields over the same region. Examples of dune fields that our model identified that were not hand-mapped are plotted above.
Next steps will involve transforming the predicted/possible dune fields back into geographic coordinates for visual inspection in GIS software. This will involve stitching image tiles together or creating polygons from the mapped dune field candidates and QC.
We have developed a deep learning workflow that utilized the UNet convolutional neural network architecture to first test the feasibility of automated detection of dune fields on Mars from satellite images. Our final model predicts the location, geometry, and extent of dune fields with a validation accuracy of 0.97. We then apply our model to the full dataset and identify areas that may correspond to additional unmapped dune fields on Mars. This project provides a proof-of-concept that CNNs may provide an automated way of locating and mapping the extent of dune fields on the Martian surface.